Keynote

Search performance evaluation can be performed using metrics based on result relevance or alternative measures based on users’ search experience. Recent studies indicate that relevance-based evaluation metrics, such as MAP and nDCG, may not be perfectly correlated with users’ search experience (usually considered as the gold standard). Therefore, search satisfaction has become one of the prime concerns in search evaluation studies. In this talk, I will discuss about some of our recent progresses in the understanding and effective prediction of search satisfaction. I will start by talking about the relationship between relevance, usefulness and satisfaction. More specifically, how do document’s usefulness perceived by the user and relevance annotated by the assessors correlate with user’s satisfaction? After that, we investigate users’ satisfaction perception in a heterogeneous search environment and try to find out how vertical results on SERPs affect users’ satisfaction. Finally, we introduce a novel satisfaction prediction framework which relies on users’ mouse movement patterns (motifs) to identify satisfied or unsatisfied search sessions.